Classification of "Near-patient" and "Point-of-Care" SARS-CoV-2 Nucleic Acid Amplification Test Systems and a first approach to evaluate their analytical independence of operator activities.

BACKGROUND: European legislation defines as "near-patient testing" (NPT) what is popularly and in other legislations specified as "point-of-care testing" (POCT). Systems intended for NPT/POCT use must be characterized by independence from operator activities during the analytic procedure. However, tools for evaluating this are lacking. We hypothesized that the variability of measurement results obtained from identical samples with a larger number of identical devices by different operators, expressed as the method-specific reproducibility of measurement results reported in External Quality Assessment (EQA) schemes, is an indicator for this characteristic. MATERIALS AND METHODS: Legal frameworks in the EU, the USA and Australia were evaluated about their requirements for NPT/POCT. EQA reproducibility of seven SARS-CoV-2-NAAT systems, all but one designated as "POCT", was calculated from variabilities in Ct values obtained from the respective device types in three different EQA schemes for virus genome detection. RESULTS: A matrix for characterizing test systems based on their technical complexity and the required operator competence was derived from requirements of the European In Vitro Diagnostic Regulation (IVDR) 2017/746. Good EQA reproducibility of the measurement results of the test systems investigated implies that different users in different locations have no recognizable influence on their measurement results. CONCLUSION: The fundamental suitability of test systems for NPT/POCT use according to IVDR can be easily verified using the evaluation matrix presented. EQA reproducibility is a specific characteristic indicating independence from operator activities of NPT/POCT assays. EQA reproducibility of other systems than those investigated here remains to be determined.

External quality assessment of SARS-CoV-2 serology in European expert laboratories, April 2021.

BackgroundCountries worldwide are focusing to mitigate the ongoing SARS-CoV-2 pandemic by employing public health measures. Laboratories have a key role in the control of SARS-CoV-2 transmission. Serology for SARS-CoV-2 is of critical importance to support diagnosis, define the epidemiological framework and evaluate immune responses to natural infection and vaccine administration.AimThe aim of this study was the assessment of the actual capability among laboratories involved in sero-epidemiological studies on COVID-19 in EU/EEA and EU enlargement countries to detect SARS-CoV-2 antibodies through an external quality assessment (EQA) based on proficiency testing.MethodsThe EQA panels were composed of eight different, pooled human serum samples (all collected in 2020 before the vaccine roll-out), addressing sensitivity and specificity of detection. The panels and two EU human SARS-CoV-2 serological standards were sent to 56 laboratories in 30 countries.ResultsThe overall performance of laboratories within this EQA indicated a robust ability to establish past SARS-CoV-2 infections via detection of anti-SARS-CoV-2 antibodies, with 53 of 55 laboratories using at least one test that characterised all EQA samples correctly. IgM-specific test methods provided most incorrect sample characterisations (24/208), while test methods detecting total immunoglobulin (0/119) and neutralising antibodies (2/230) performed the best. The semiquantitative assays used by the EQA participants also showed a robust performance in relation to the standards.ConclusionOur EQA showed a high capability across European reference laboratories for reliable diagnostics for SARS-CoV-2 antibody responses. Serological tests that provide robust and reliable detection of anti-SARS-CoV-2 antibodies are available.

Results of a European-Wide External Quality Assessment (EQA) Scheme for Serological Detection of Anti-SARS-CoV-2 (CoVimm)-Pitfalls of Routine Application.

BACKGROUND: During the last two years, a variety of assays for the serological detection of antibodies to the new SARS-CoV-2 virus have been launched and used as part of standard care in many laboratories. The pace with which these tests have been introduced into routine care emphasizes the importance of quality measures for analytical methods, particularly with regard to the implications of results for clinical and epidemiologic decisions. Accuracy, reliability and comparability of analytical test results are thus essential, and here external quality assessment (EQA) is the most important quality assurance tool. It allows us to achieve harmonization of test methods as a prerequisite for a high standard of performance for laboratory and analytical techniques and their interpretation. METHODS: This EQA scheme consisted of pre-characterized clinical biospecimens dedicated to the analysis of anti-SARS-CoV-2 IgG total antibodies and differentiation into spike protein-specific IgG antibodies against SARS-CoV-2 (anti-S-SARS-CoV-2) and nucleocapsid-specific IgG antibodies against SARS-CoV-2 (anti-N-SARS-CoV-2). RESULTS: A total of 239 laboratories across Europe participated in this scheme, called CoVimm. In detail, 536 results for anti-SARS-CoV-2 IgG, 431 results for anti-S-SARS-CoV-2 IgG, and 200 results for anti-N-SARS-CoV-2 IgG were reported. Based on the pre-defined thresholds, the success rates for the determination of anti-S-SARS-CoV-2 IgG and anti-N-SARS-CoV-2 IgG were 96% and 90%, respectively. Interestingly, only 64% of the participating laboratories successfully passed the EQA scheme for the determination of total anti-SARS-CoV-2 IgG. CONCLUSIONS: This EQA revealed serious concerns regarding the reliability and appropriate use of anti-SARS-CoV-2 antibody assays in routine care. In addition to the wide heterogeneity of different assays used by participating laboratories, a lack of standardization and harmonization is also evident. This is of particular importance for reliable and clinically meaningful interpretation of test results.

First Results of an External Quality Assessment (EQA) Scheme for Molecular, Serological and Antigenic Diagnostic Test for SARS-CoV-2 Detection in Lombardy Region (Northern Italy), 2020-2022.

For diagnosing SARS-CoV-2 infection and for monitoring its spread, the implementation of external quality assessment (EQA) schemes is mandatory to assess and ensure a standard quality according to national and international guidelines. Here, we present the results of the 2020, 2021, 2022 EQA schemes in Lombardy region for assessing the quality of the diagnostic laboratories involved in SARS-CoV-2 diagnosis. In the framework of the Quality Assurance Programs (QAPs), the routinely EQA schemes are managed by the regional reference centre for diagnostic laboratories quality (RRC-EQA) of the Lombardy region and are carried out by all the diagnostic laboratories. Three EQA programs were organized: (1) EQA of SARS-CoV-2 nucleic acid detection; (2) EQA of anti-SARS-CoV-2-antibody testing; (3) EQA of SARS-CoV-2 direct antigens detection. The percentage of concordance of 1938 molecular tests carried out within the SARS-CoV-2 nucleic acid detection EQA was 97.7%. The overall concordance of 1875 tests carried out within the anti-SARS-CoV-2 antibody EQA was 93.9% (79.6% for IgM). The overall concordance of 1495 tests carried out within the SARS-CoV-2 direct antigens detection EQA was 85% and it was negatively impacted by the results obtained by the analysis of weak positive samples. In conclusion, the EQA schemes for assessing the accuracy of SARS-CoV-2 diagnosis in the Lombardy region highlighted a suitable reproducibility and reliability of diagnostic assays, despite the heterogeneous landscape of SARS-CoV-2 tests and methods. Laboratory testing based on the detection of viral RNA in respiratory samples can be considered the gold standard for SARS-CoV-2 diagnosis.

A localized small-scale external quality assessment (EQA) for PCR testing of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in the molecular laboratories.

The aim of this study was to estimate the PCR results for SARS-CoV-2 testing in 32 participating laboratories in a localized small-scale external quality assessment (EQA) scheme. EQA samples were distributed to the participants and detected immediately on the day of delivery. Qualitative results were submitted to the EQA provider, including negative or positive results along with cycle threshold (Ct) values for different target genes. Although the variability of Ct values differed among the laboratories in the EQA, a total of 32 (100 %) participants reported correct qualitative results. The study showed that the mean loads of N or E gene were higher than those of ORF1ab in SARS-CoV-2 RNA samples. Regardless of the analyzed gene target, the mean Ct values for weak positive and positive samples varied by fewer than 1.74 and 1.91 cycles, respectively. Less than 12 % of reported Ct values for ORF1ab and N genes deviated by more than ±4 cycles (maximum: -9.92 cycles), while none deviated by more than ±4 cycles for the E gene. The current EQA program can provide a robust practical basis for follow-up planning to conduct evaluations for SARS-CoV-2 PCR testing and other novel emerging pathogens in the future.

Inadequate design of mutation detection panels prevents interpretation of variants of concern: results of an external quality assessment for SARS-CoV-2 variant detection.

OBJECTIVES: Mutation-specific PCR assays have quickly found their way into laboratory diagnostics due to their capacity to be a fast, easy to implement and high-throughput method for the detection of known SARS-CoV-2 variants of concern (VoCs). However, little is known about the performance of such assays in routine laboratory analysis. METHODS: The results reported in a recent round of an external quality assessment (EQA) scheme for SARS-CoV-2 mutation-specific PCR were retrospectively analyzed. For the determination of individual variant-specific sequences as well as for the interpretation results for certain virus variants, correct, incorrect, and unreported results were evaluated, and their possible causes were investigated. RESULTS: A total of 34 laboratories participated in this study. For five samples containing the VoC Alpha + E484K, Beta, Gamma, Delta, or B.1.1.318 (as a variant of interest), 848 results for SARS-2-CoV mutation detection were reported, 824 (97.2%, range per sample 88-100%) of which were correct. Melting curve assays gave 99% correct results, real-time RT-qPCR 94%, microarray-based assays 100%, and MALDI-TOF MS 96%. A total of 122/167 (73%) reported results for SARS-CoV-2 variant determination were correct. Of the 45 inconclusive or incorrect results, 33 (73%) were due to inadequate selection of targets that did not allow identification of contemporary VoC, 11 (24%) were due to incorrect results, and one (3%) was due to correct results of mutation-specific PCR. CONCLUSIONS: Careful and up-to-date selection of the targets used in mutation-specific PCR is essential for successful detection of current SARS-CoV-2 variants.

Assessing the Quality of Serological Testing in the COVID-19 Pandemic: Results of a European External Quality Assessment (EQA) Scheme for Anti-SARS-CoV-2 Antibody Detection.

External quality assessment (EQA) is a key instrument for achieving harmonization, and thus a high quality, of diagnostic procedures. As reliable test results are crucial for accurate assessment of SARS-CoV-2 infection prevalence, vaccine response, and immunity, and thus for successful management of the ongoing COVID-19 pandemic, the Reference Institute for Bioanalytics (RfB) was the first EQA provider to offer an open scheme for anti-SARS-CoV-2 antibody detection. The main objectives of this EQA were (i) to gain insights into the current diagnostic landscape and the performance of serological tests in Europe and (ii) to provide recommendations for diagnostic improvements. Within the EQA, a blinded panel of precharacterized human serum samples with variable anti-SARS-CoV-2 antibody titers was provided for detection of anti-SARS-CoV-2 IgG, IgA, and IgM antibodies. Across the three distribution rounds in 2020, 284 laboratories from 22 countries reported a total of 3,744 results for anti-SARS-CoV-2 antibody detection using more than 24 different assays for IgG. Overall, 97/3,004 results were false for anti-SARS-CoV-2 IgG, 88/248 for IgA, and 34/124 for IgM. Regarding diagnostic sensitivity and specificity, substantial differences were found between the different assays used, as well as between certified and noncertified tests. For cutoff samples, a drop in the diagnostic sensitivity to 46.3% and high interlaboratory variability were observed. In general, this EQA highlights the current variability of anti-SARS-CoV-2 antibody detection, technical limitations with respect to cutoff samples, and the lack of harmonization of testing procedures. Recommendations are provided to help laboratories and manufacturers further improve the quality of anti-SARS-CoV-2 serological diagnostics.

The versatility of external quality assessment for the surveillance of laboratory and in vitro diagnostic performance: SARS-CoV-2 viral genome detection in Austria.

OBJECTIVES: External quality assessment (EQA) schemes provide information on individual and general analytical performance of participating laboratories and test systems. The aim of this study was to investigate the use and performance of SARS-CoV-2 virus genome detection systems in Austrian laboratories and their preparedness to face challenges associated with the pandemic. METHODS: Seven samples were selected to evaluate performance and estimate variability of reported results. Notably, a dilution series was included in the panel as a measure of reproducibility and sensitivity. Several performance criteria were evaluated for individual participants as well as in the cohort of all participants. RESULTS: A total of 109 laboratories participated and used 134 platforms, including 67 different combinations of extraction and PCR platforms and corresponding reagents. There were no false positives and 10 (1.2%) false negative results, including nine in the weakly positive sample (Ct ∼35.9, ∼640 copies/mL). Twenty (22%) laboratories reported results of mutation detection. Twenty-five (19%) test systems included amplification of human RNA as evidence of proper sampling. The overall linearity of Ct values from individual test systems for the dilution series was good, but inter-assay variability was high. Both operator-related and systematic failures appear to have caused incorrect results. CONCLUSIONS: Beyond providing certification for participating laboratories, EQA provides the opportunity for participants to evaluate their performance against others so that they may improve operating procedures and test systems. Well-selected EQA samples offer additional inferences to be made about assay sensitivity and reproducibility, which have practical applications.

Variable Sensitivity of SARS-CoV-2 Molecular Detection in European Expert Laboratories: External Quality Assessment, June and July 2020.

During the ongoing coronavirus disease 2019 (COVID-19) outbreak, robust detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is a key element for clinical management and to interrupt transmission chains. We organized an external quality assessment (EQA) of molecular detection of SARS-CoV-2 for European expert laboratories. An EQA panel composed of 12 samples, containing either SARS-CoV-2 at different concentrations to evaluate sensitivity or other respiratory viruses to evaluate specificity of SARS-CoV-2 testing, was distributed to 68 laboratories in 35 countries. Specificity samples included seasonal human coronaviruses hCoV-229E, hCoV-NL63, and hCoV-OC43, as well as Middle East respiratory syndrome coronavirus (MERS-CoV), SARS-CoV, and human influenza viruses A and B. Sensitivity results differed among laboratories, particularly for low-concentration SARS-CoV-2 samples. Results indicated that performance was mostly independent of the selection of specific extraction or PCR methods.

The challenge of producing an EQA for the COVID-19 pandemic.

The COVID-19 pandemic has changed the clinical medicine landscape. The importance of pathology testing has come to the forefront. Patients or potential patients are dealing directly with laboratories as they line up in carparks or testing staff come to the front doors to obtain samples. Laboratories have had to increase capacity to deal with the high volumes of testing driven by the need to identify and quarantine cases. Supporting this effort, External Quality Assurance scheme providers have also needed to produce COVID-19 Proficiency Testing (PT) programs which are fit for purpose. COVID-19 Point of Care testing has become critical frontline testing and has required the PT programs to be simple to use, readily accessible and robust. We describe a COVID-19 PoCT Serology PT program supported by a mobile phone App. The App is described, and the advantages made explicit. This App suggests that the way that PoCT EQA/PT programs may be deployed in the future.

International external quality assessment for SARS-CoV-2 molecular detection and survey on clinical laboratory preparedness during the COVID-19 pandemic, April/May 2020.

Laboratory preparedness with quality-assured diagnostic assays is essential for controlling the current coronavirus disease (COVID-19) outbreak. We conducted an external quality assessment study with inactivated severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) samples to support clinical laboratories with a proficiency testing option for molecular assays. To analyse SARS-CoV-2 testing performance, we used an online questionnaire developed for the European Union project RECOVER to assess molecular testing capacities in clinical diagnostic laboratories.